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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 10 April 2000 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-22 is/are pending In the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \Z\ Claim(s) is/are allowed. 

6) K Claim(s) 1-22 is/are rejected. 

Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) K The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 10 April 2000 is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
11 )□ The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) K Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)IElAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) O Inten/iew Summary (PTO-413) Paper No(s). . 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) [U Notice of Informal Patent Application (PTO-1 52) 

3) K Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 8. 6) O Other: 
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DETAILED ACTION 



Priority 



1 . Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which papers 
have been placed of record in the file. 



2. Figure 4 (page 8, line 5-7) and figures 8-10 (page 8, line 25 to page 9, line 2) should be 
designated by a legend such as -Prior Art- because only that which is old is illustrated. See 
MPEP § 608.02(g). A proposed drawing correction or corrected drawings are required in reply 
to the Office action to avoid abandonment of the application. The objection to the drawings will 
not be held in abeyance. 



3. The disclosure is objected to because of the following informalities: 

• Page 9, line 15 states that the scanning operation is shown in Figure 7, but this 
operation is shown in Figure 8. 

• Page 9, line 21 refers to RAM with reference numeral 38, but figure 4 shows this as 
reference numeral 39. 

• Page 20, line 7-8 refers to non-volatile memory with reference numeral 37, but figure 
4 shows this as reference numeral 38. 

• Page 21, line 5 refers to processor with reference numeral 1, but figure 4 shows this 
as reference numeral 3 1 . 

• Page 22, line 1 refers to a camera with reference numeral 20, but figure 1 shows two 
cameras, one with reference numeral 21 , and one with reference numeral 22. 



Drawings 



Specification 
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Appropriate correction is required. 

Claim Objections 

4. Claims 2 and 13 are objected to because of the following informalities: line 2 states, 
"reference models are obtained form a part of the image." Applicant apparently intended this to 
be "obtained from a part of the image." Appropriate correction is required. 

5. Claim 6 is objected to because of the following informalities: Claim 6 does not end with 
a period. Appropriate correction is required. 

6. Claims 1 1 and 22 are objected to because of the following informalities: line 2-3 states, 
"picking up at least one object form a plurality of objects." Applicant apparently intended this to 
be "object from a plurality of objects." Appropriate correction is required. 

Claim Rejections - 35 USC § 112 

7. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

8. Claims 8-10 and 19-21 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claims 8-10 and 19-21 recite the limitation "said second 
image data capturing device." There is insufficient antecedent basis for this limitation in the 
claim. 

Claim Rejections - 35 USC §102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 



10. Claims 1-4 and 12-15 are rejected under 35 U.S.C. 102(b) as being anticipated by Suzuki 
(USPN 4,835,450). 

1 1 . Regarding claim 1 , Suzuki discloses a robot system having an image processing function 
for determining posture, or posture and position of a robot operation on an object (column 4, line 
25-39) comprising: a robot (figure 1, element 3); a first image capturing device (figure 1, 
element 1 la); a memory storing reference models (figure 2, element 19) created based on image 
data of a reference object captured by said image capturing device in a plurality of different 
directions, and storing information of the capturing directions to be respectively associated with 
said reference models, and information of orientation of the robot operation with respect to the 
object (column 3, line 66 to column 4, line 9), said reference object (figure 1, element 10) being 
the object of detection or an object having a shape identical (figure 1, element 15) to that of the 
object of detection (column 3, line 57-62); and a processor (figure 2, element 18) to perform 
matching processing on image data containing an image of the object of detection captured by 
said image capturing device with said reference models to select an image of an object matched 
with one of said reference models (column 4, line 17-24), and to determine orientation, or 
orientation and position of an operation to be performed by the robot based on the selected image 
of the object, said one reference model and the information of the capturing direction and the 
information of the orientation of the robot operation with respect to the object associated with 
said one reference model (column 5, line 5-11). 

12. Regarding claim 12, which is substantially the same as claim 1 except that claim 12 
includes the limitation that the memory stores reference models created based on image data of 



f 
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different kinds of reference objects, Suzuki discloses a robot system having an image processing 
function including a memory storing reference models created based on image data of different 
kinds of reference objects captured by said first image capturing device (column 4, line 17-24). 
See discussion of claim 1 above for all other common features. 

13. Regarding claims 2 and 13, Suzuki discloses that said reference models are obtained from 
a part of the image data of the reference object (column 4, line 2-9). 

14. Regarding claims 3 and 14, Suzuki discloses that said reference models are obtained by 
processing the image data of the reference object (column 4, line 5-16). 

15. Regarding claims 4 and 15, Suzuki discloses that said first image capturing device 
comprises a camera for capturing two-dimensional image data (column 3, line 66 to column 4, 
line 5). 



16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



17. Claims 5 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Suzuki 
as applied to claims 1 and 12 above, and further in view of Suyama et al. (USPN 4,879,664), 
hereafter Suyama, or Stauffer (USPN 4,410,804). 

1 8. Suzuki does not disclose that the image data are captured from a predetermined distance. 
Suyama discloses a three-dimensional position sensor comprising robot teaching apparatus 
wherein said image data of the reference object are captured by said camera (figure 11a, element 



Claim Rejections - 35 USC §103 
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35) from a predetermined distance (column 6, line 20-37). In addition, Stauffer teaches that if a 
two-dimensional sensor is used, the image processor is unable to simultaneously determine the 
distance to the object unless the objects are always positioned at a known distance (column 1, 
line 31-39). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to capture an image at a predetermined distance in order to calibrate a three- 
dimensional sensor as taught by Suyama or such that the distance to the object need not be 
determined by other means as taught by Stauffer. 

19. Claims 6, 7, 17, and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Suzuki as applied to claims 1 and 12 above, and further in view of Maeno et al. (USPN 
5,047,714), hereafter Maeno. 

20. Regarding claims 6 and 17, Suzuki discloses a step of determining the orientation and/or 
position of the object and moving the robot arms to have said orientation and/or position, but 
does not disclose that the robot moves a second image capture device to have the determined 
orientation and/or position. Maeno discloses that said robot system further comprises a second 
image capturing device (figure 7c, element 10) and said robot situates said second image data 
capturing device to have said determined position (column 4, line 47-49) with respect to the 
object, and said processor processes second image data captured by said second image capturing 
device to detect position and/or posture of the object with respect to said second image data 
capturing device (column 4, line 54-63). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to move the second image capture device to have the 
determined orientation and/or position as taught by Maeno in order to provide fine-tuning 
adjustments to the orientation and/or position determined by the first image capture device. 
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21 . Regarding claims 7 and 18, Suzuki discloses that said robot system further comprises a 
second image capturing device (figure 1, element 1 lb) for obtaining three-dimensional position 
(column 4, line 5-16) and that said second image data capturing device is directed to a 
characterizing portion of the object (column 3, line 66 to column 4, line 5), but does not disclose 
that the robot moves the second image capture device to have the determined orientation and/or 
position. Maeno discloses that said robot situates said second image data capturing device to 
have said determined position with respect to the object; and said processor processes second 
image data captured by said second image capturing device to detect position and/or posture of 
the object with respect to said second image data capturing device (see discussion of claims 6 
and 17 above). 

22. Claims 8-10 and 19-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Suzuki as applied to claims 1 and 12 above, and further in view of Sakakibara et al. (JP 07- 
270137), hereafter Sakakibara, English language translation included. 

23. Regarding claims 8-10 and 19-21, see 35 U.S.C. 1 12 second paragraph rejection above. 

24. Regarding claims 9 and 20, Suzuki does not disclose a second image capturing device 
comprising a three dimensional visual sensor of spot-light scarming type. Sakakibara discloses a 
three dimensional visual sensor usable in robot automation (page 1, paragraph 1), of spot-light 
scanning type capable of measuring distance between the sensor and an object (page 3, 
paragraph 1 5). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to utilize a three dimensional visual sensor of spot-light scanning type as 
taught by Sakakibara in order to accurately determine the three dimensional position of an object 
using one device. 
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25. Regarding claims 10 and 21, Suzuki does not disclose a second image data capturing 
device comprising a structured-light unit for irradiating a structured light on an object. 
Sakakibara discloses an image data capturing device comprising a structured-light unit for 
irradiating a structured light on an object and capturing an image of the object including the 
irradiated light on the object (page 3, paragraph 16). It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to utilize a structured light unit as 
taught by Sakakibara in order to accurately determine the three dimensional position of an object 
using one device. 

26. Claims 1 1 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Suzuki 
in view of Maeno as applied to claims 7 and 18 above, and further in view of Kelley et al. 
(USPN 4,305,130), hereafter Kelley. 

27. Neither Suzuki nor Maeno expressly disclose that said robot operation is an operation of 
picking up at least one object from a plurality of objects overlapped with each other. Kelley 
discloses an apparatus and method to enable a robot with vision to acquire, orient and transport 
workpieces including a robot operation of picking up at least one object from a plurality of 
objects overlapped with each other (figure 9; column 13, line 6-15). It would have been obvious 
to one of ordinary skill in the art at the time the invention was made to pick objects from a 
plurality of overlapping objects as taught by Kelley in order to retrieve randomly oriented parts 
from a bin to save time by automatically orienting the parts for manufacturing or the like 
(column 1, line 12-32). 

Conclusion 
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28. The prior art made of record and not relied upon is considered pertinent to applicant's 



disclosure. 



• USPN 4,61 1 ,292 to Ninomiya et al. discloses a robot vision system comprising a 
three dimensional vision sensor. 

• USPN 6,135,854 to Masumura et al. discloses an automatic workpiece transport 
apparatus for double-side polishing machine. 

• USPN 4,876,728 to Roth discloses a vision system for distinguishing touching parts. 
Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-872-9314 for regular 
communications and 703-872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-306-0377. 



rjh 

January 16, 2003 




